KN YUR RE

Ryan Transfiguracion

July 23, 2018

Outline

  • Background
  • Team Roles
  • The Problem
  • The Data
  • Exploratory Analysis
  • Machine Learning Models
  • Conclusion
  • The Future and Recommendations

Background

What is League of Legends?

  • A free-to-play, multiplayer online battle arena (MOBA) videogame
  • Developed and published by Riot Games
  • ~100 million active players a month
  • ~100k viewers during peak hours on Twitch.tv
  • Active and popular professional eSports scene
  • Numerous 3rd party websites and books on gameplay strategies

How is League of Legends Played?

  • Primary mode: Summoner’s Rift
  • 5 vs. 5 human players, plus AI minions for each team
  • Main objective: destroy enemy’s base (aka The Nexus)
  • Sub-objectives: destroy enemy’s defense turrents
  • Key objectives: slay dragons and Baron Nashor for boosts

How is League of Legends Played?

  • Players select 1 of 141 champions to represent them
  • Each champion has unique set of attributes and abilities
  • Gain power by accumulating: XP to level up, and gold to buy enhancement items
  • Accumulate XP/gold by killing minions, monsters, and enemy players, and taking objectives
  • Killed players must wait before respawn

Team Roles

What are Team Roles?

  • In standard team composition, each player is designated one of fives roles based on:
  • Their champion’s attributes and abilities
  • Their intended location on the map near start of a match (around 1:25-1:30) – called the laning phase
  • The Roles: Top Laner, Jungler, Mid Laner, Botlane Carry, Support

Top Laner

  • Fights in the top lane of the map during laning phase
  • Key attributes/responsibilities: front line tankiness, deal moderate-to-high damage, ability to safely initiate team fights

Jungler

  • Fights in the jungle areas of the map during laning phase
  • Key attributes/responsibilities: ability to efficiently kill most jungle monsters during laning phase, provide ganks for in-lane teammates, place wards on map
  • Can deal high burst damage to neutral monsters via Smite

Mid Laner

  • Fights in the middle lane of the map during laning phase
  • Key attribute/responsibility: deal high burst, ability power-based damage
  • Champion is usually a mage or an assassin

Botlane Carry

  • Fights in the bottom lane of the map during laning phase
  • Key attributes/responsibilities: back line squishiness, deal high sustained attack damage, be the highest damage dealer in match
  • Champion is usually ranged (e.g., guns, arrows, etc.)

Support

  • Fights/assists in the bottom lane of the map during laning phase
  • Key attributes/responsibilities: help Botlane Carry survive and achieve kills, provide healing, shielding, and/or other utilities during team fights, place wards on map

The Problem

The Problem

  • Humanly impossible to visually observe every match and record team roles manually.
  • Can we use match data and machine learning to distinguish all five team roles from each other?

Team Roles – Match Overview, eSports

Assumed Team Roles – eSports Match

Assumed Team Role Participant IDs – Draft Pick Match
TeamRole BlueTeam RedTeam
TOP 1 6
JUNGLE 2 7
MID 3 8
BOTCARRY 4 9
SUPPORT 5 10

Explicit Team Roles – Draft Pick Mode Selection Screen

Team Roles – Match Overview, Blind Pick Format

Team Roles – NA LCS 2018 Spring Split, All Matches

Predicted Roles (Rows) vs. Actual/Assumed Roles (Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
DUO, BOTTOM 77 0 57 77 0
DUO, MIDDLE 0 55 0 0 55
DUO_CARRY, BOTTOM 111 0 0 0 0
DUO_CARRY, MIDDLE 0 0 4 0 0
DUO_SUPPORT, BOTTOM 0 0 0 111 0
DUO_SUPPORT, MIDDLE 0 4 0 0 0
NONE, JUNGLE 9 173 58 8 57
SOLO, BOTTOM 5 0 0 38 0
SOLO, MIDDLE 0 2 115 0 0
SOLO, TOP 32 0 0 0 122

Why Should We Care?

  • Most LoL analytics websites only track Draft Pick mode matches because of extracting explicit team roles
  • Some only track Ranked (ladder) matches
  • Normal Draft Pick requires owning/having available at least 20 champions
  • Ranked Draft Pick requires owning at least 20 champions plus being Gamer/Summoner Level 30
  • Granting sites ability to analyze team role performace in Blind Pick matches adds another avenue of engagement for newcomers/casuals in order to graduate to dedicated/hardcore
  • Provide additional team role-focused metrics when testing new/reworked champions

The Data

Data Overview

  • Data for all matches in the 2018 Spring Split seasons of the North America LoL Championship Series (NALCS), the Europe LoL Championship Series (EULCS), LoL Champions Korea (LCK), and LoL Master Series (LMS), plus the 2018 Mid-Season Invitational (MSI)
  • Each match data accessed using API domain https://acs.leagueoflegends.com/v1/stats/game/

Synopsis of Individual Match Player Datasets

  • NA LCS: 117 matches, 1170 observations
  • EU LCS: 115 matches, 1150 observations
  • LCK: 246 matches, 2460 observations
  • LMS: 144 matches, 1440 observations
  • MSI: 74 matches, 740 observations

Synopsis of Per-Match Average Datasets

  • NA LCS: 56 players
  • EU LCS: 57 players
  • LCK: 81 players
  • LMS: 69 players
  • MSI: 74 players
  • (More technical details below…)

Obtaining API and Match IDs

Sample Collection of Match IDs

Match ID Data, First Six Matches of NA LCS 2018 Spring Split
Region.ID Game.ID Hash.ID Blue.Team Red.Team
TRLH1 1002440062 a3b08c115923f00d Team Liquid Team Solo Mid
TRLH1 1002440076 c426d3d50426edb3 100 Thieves OpTic Gaming
TRLH1 1002440084 f0f86e52b6e472e9 Clutch Gaming Golden Guardians
TRLH1 1002440095 fd3b9331ff5312e3 Echo Fox FlyQuest
TRLH1 1002440106 d6441d4ec8f87534 Counter Logic Gaming Cloud9
TRLH1 1002440127 361bcc2e848641d2 OpTic Gaming Team Liquid

Cleaning Each Match for Player Stats

  • JSON response object contains two sub-objects (participants and participantIdentities) that contain indvidual player stats
  • participantIdentities contains 10 players’ usernames under sub-object player
  • participants contains two more sub-objects (stats and timeline)
  • basically, participants is flattened out, then joined with player
  • final product for single match is 10 rows and nearly 150 columns/variables
  • process repeated for all other matches, and tables are concatenated together

Sample of MSI 2018 Match Player Dataset

summonerName kills deaths assists totalDamageTaken
403 EVS Warzone 1 3 5 11648
683 KZ Bdd 3 1 9 7511
732 RNG Karsa 1 3 10 19421
464 RNG Uzi 6 1 10 9316
539 EVS Slay 2 4 2 11608
39 KLG Fix 3 2 10 9255
41 KLG Nate 2 1 4 20259
406 KZ Khan 3 0 4 19232
565 EVS RonOP 0 4 2 18872
157 GMB Diamondprox 1 2 7 24735
59 GMB Lodik 5 0 7 5534
117 SUP Stomaged 2 0 15 14868

Creating Player Per-Match Average Datasets

  • Observations from the match player datasets are grouped together by team, player name, and then assumed team role
  • Each group’s variables’ statistics are calculated to mean (average) values.

Sample of MSI 2018 Player Per-Match Average Dataset

summonerName kills deaths assists totalDamageTaken
65 RNG Ming 0.8888889 1.500000 8.888889 11705.83
24 FW Moojin 2.1666667 1.666667 5.222222 24486.17
63 RNG Karsa 1.5714286 1.571429 8.714286 22081.57
33 GMB Edward 0.8888889 2.666667 7.222222 14253.56
70 TL Impact 1.7272727 2.272727 4.000000 23604.55
66 RNG Mlxg 2.4545455 1.818182 5.454546 26075.09
50 KZ Khan 2.3888889 2.055556 3.222222 22908.28
36 GMB PVPStejos 1.5555556 2.222222 5.333333 29927.33
14 DW Shernfire 2.1666667 3.666667 4.166667 29793.67
10 SUP Zeitnot 5.2000000 2.200000 4.900000 10362.70
18 EVS Stark 2.0000000 2.142857 4.571429 19437.71
16 EVS RonOP 0.5714286 4.214286 5.071429 17344.43

Exploratory Analysis

Box Plots 1

Box Plots 2

Box Plots 3

Box Plots 4

Machine Learning Models

Feature Set for Machine Learning 1

variableName description
kills The number of enemy champions killed.
assists The number of enemy champions assisted in killing.
magicDamageDealt The amount of magic damage dealt.
physicalDamageDealt The amount of physical damage dealt.
magicDamageDealtToChampions The amount of magic damage dealt to enemy champions only.
physicalDamageDealtToChampions The amount of physical damage dealt to enemy champions only.
totalHeal The amount of health points the player has regained.

Feature Set for Machine Learning 2

variableName description
totalUnitsHealed The number of entities a player healed.
damageSelfMitigated The amount of health points that were not lost from damage.
totalDamageTaken The amount of damage a player took from various sources.
neutralMinionsKilled The number of neutral monsters killed by a player.
timeCCingOthers The weighted sum of all CC applied
totalTimeCrowdControlDealt The sum of all CC applied
champLevel The (experience) level of a player at the end of a match.

Feature Set for Machine Learning 3

variableName description
visionWardsBoughtInGame The number of wards (i.e. surveillance items) a player purchased.
wardsPlaced The number of wards a player placed in the arena.
wardsKilled The number of enemy wards a player destroyed.
physDmgToChampsToDmgTakenRatio Physical damage dealt to champions / Total damage taken
totalMinionsKilled The number of enemy minions killed.
damageTakenPerMinDeltas.0.10 The amount of damage taken per minute from 0:00 to 10:00 minutes.
creepsPerMinDeltas.0.10 The number of enemy minions killed per minute from 0:00 to 10:00 minutes.

Machine Learning Approach

  1. Put together all five datasets for player per-match averages, then split the entire set into two subsets: one for training and one for testing.
  2. Create/train a model using the k-means clustering algorithm on the training set. Since we already know how many team roles exist, we set k=5.
  3. Test/validate the model using the k-nearest-neighbor (knn) classification algorithm on the testing set. Since there are only five possible neighbors, and we want to classify each observation to exactly one group (team role), we set k=1.

Machine Learning Approach (continued)

  1. Put together all five datasets for single match player stats.
  2. Test the model using knn on the new dataset.
  3. Evaluate performance of Steps 2, 3, and 5 using a confusion matrix.

Note: features are scaled using z-score.

Training the Model

k-means: Creating Training Model (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 18 0 0 0 0
JUNGLE 0 22 0 0 0
MID 0 0 20 0 0
SUPPORT 0 0 0 17 0
TOP 0 0 0 0 19

Validating the Model

knn: Predict Team Roles w/Per-Game Averages Test Data Set (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 27 0 0 0 0
JUNGLE 0 31 0 0 0
MID 0 0 28 0 0
SUPPORT 0 0 0 24 0
TOP 0 0 0 0 27

Testing the Model (Individual Matches Dataset)

knn: Predict Team Roles Single Games Test Data Set (Predicted Rows vs. Actual Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
BOTCARRY 1321 1 10 0 68
JUNGLE 0 1378 1 14 4
MID 19 1 1321 0 120
SUPPORT 0 7 1 1376 0
TOP 52 5 59 2 1200

Confusion Matrix

## Confusion Matrix and Statistics
## 
##                           
## knn_pred_test_single_games BOTCARRY JUNGLE  MID SUPPORT  TOP
##                   BOTCARRY     1321      1   10       0   68
##                   JUNGLE          0   1378    1      14    4
##                   MID            19      1 1321       0  120
##                   SUPPORT         0      7    1    1376    0
##                   TOP            52      5   59       2 1200
## 
## Overall Statistics
##                                           
##                Accuracy : 0.9477          
##                  95% CI : (0.9422, 0.9528)
##     No Information Rate : 0.2             
##     P-Value [Acc > NIR] : < 2.2e-16       
##                                           
##                   Kappa : 0.9346          
##  Mcnemar's Test P-Value : NA              
## 
## Statistics by Class:
## 
##                      Class: BOTCARRY Class: JUNGLE Class: MID
## Sensitivity                   0.9490        0.9899     0.9490
## Specificity                   0.9858        0.9966     0.9749
## Pos Pred Value                0.9436        0.9864     0.9042
## Neg Pred Value                0.9872        0.9975     0.9871
## Prevalence                    0.2000        0.2000     0.2000
## Detection Rate                0.1898        0.1980     0.1898
## Detection Prevalence          0.2011        0.2007     0.2099
## Balanced Accuracy             0.9674        0.9933     0.9619
##                      Class: SUPPORT Class: TOP
## Sensitivity                  0.9885     0.8621
## Specificity                  0.9986     0.9788
## Pos Pred Value               0.9942     0.9105
## Neg Pred Value               0.9971     0.9660
## Prevalence                   0.2000     0.2000
## Detection Rate               0.1977     0.1724
## Detection Prevalence         0.1989     0.1894
## Balanced Accuracy            0.9935     0.9204

Current Classification System

Predicted Roles (Rows) vs. Actual/Assumed Roles (Columns)
BOTCARRY JUNGLE MID SUPPORT TOP
DUO, BOTTOM 538 0 395 538 0
DUO, MIDDLE 0 347 0 0 347
DUO, TOP 0 3 0 0 3
DUO_CARRY, BOTTOM 533 0 0 0 0
DUO_CARRY, MIDDLE 0 0 12 0 0
DUO_SUPPORT, BOTTOM 0 0 0 533 0
DUO_SUPPORT, MIDDLE 0 12 0 0 0
NONE, JUNGLE 91 985 402 66 397
SOLO, BOTTOM 15 0 0 255 0
SOLO, MIDDLE 0 23 583 0 22
SOLO, TOP 215 22 0 0 623

Conclusion

Summary

  • 21 out of ~150 variables were chosen for model’s feature set
  • k-means clustering on per-match averages training set was 100% accurate
  • Used centroids produced from clustering as the model for validation and testing sets
  • Model used for knn classification on per-match averages validation set was 100% accurate
  • Used same model on “real world” single match performances test set
  • knn classification on test set was 94.7% accurate

Caveats / Limitations

  • Sample size vs. population
  • Patches ==> possible meta changes
  • Normal / Blind Pick / vs. AI modes are lower stakes
  • Unique champions (e.g. Vladimir, Swain, Gangplank, Ivern)
  • Varying levels of competition

The Future and Recommendations

Application of Solution

  • Train model by running k-means on sample of Draft Pick mode matches, then run knn on subsequent matches of any game mode to classify team roles
  • When creating new champions or reworking current ones, while in test, can run the model to see if champion is performing certain team roles as expected

Future Refinements

  • Larger datasets: virtually endless number of non-custom matches can be accessed off of one username alone due to recursive nature of accessing match history
  • Additional data (i.e., the timeline API using match IDs)
  • Alternative k-means training
  • Creating models per division
  • Creating models per patch

Final Thoughts

  • Draft Pick game modes allow safe assumptions on team roles
  • Limits the number of matches analytics sites can use for analyzing champion and team role performances
  • While not absolutely perfect, model could provide excellent tool for classifying team roles in Blind Pick Summoner’s Rift game modes,where team roles cannot be assumed based on Participant ID alone

The End